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VIRTUAL  REALITY  FEATURES  OF  FRAME  OF  REFERENCE  AND 
DISPLAY  DIMENSIONALITY  WITH  STEREOPSIS:  THEIR  EFFECTS  ON 

SCIENTIFIC  VISUALIZATION 


ABSTRACT 

Initial  discussion  reviews  the  features  of  VR  and  their  possible  effects  of 
scientific  visualization  performance.  Three  of  these  features:  Dimensionality, 
stereopsis,  and  frame  of  reference  are  modified  in  an  experiment  that  contrasted 
performance  using  a  2D  display,  and  four  displays  varying  in  frame  of  reference 
(immersed  vs.  non-immersed)  and  in  the  presence  of  stereoscopic  vision. 
Performance  was  measured  across  four  separate  scientific  visualization  subtasks: 
Search,  Travel,  Local  judgment  support,  and  Global  judgment  support. 

Participants  were  instructed  to  locate  and  follow  a  designated  path  through  simple 
virtual  environments  and  to  answer  questions  about  that  environment.  Subjects 
were  divided  into  two  groups  varying  in  the  type  of  frame  of  reference  used;  either 
an  immersive  or  non-immersive.  Each  subject  was  to  complete  5  trials  in  a  2D 
condition,  5  trials  with  their  particular  frame  of  reference  presented  in  monoscopic 
vision,  and  5  trials  with  their  particular  frame  of  reference  presented  in  stereo.  The 
results  revealed  that  2D  performance  was  substantially  worse  than  3D  performance 
across  both  frames  of  reference  and  stereo  conditions.  The  results  also  indicate 
that  the  immersed  frame  of  reference  supported  better  travel  performance,  but 
severely  hampered  search  and  global  judgment  support.  Stereoscopic  vision 
slowed  search  performance  and  only  enhanced  travel  using  the  non-immersed 
frame  of  reference.  Local  judgment  accuracy  benefited  from  stereo,  but  response 
time  was  unaffected.  Global  judgments  showed  no  accuracy  advantage  from 
stereo,  but  did  show  faster  responses  when  stereo  was  used. 
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INTRODUCTION 


The  rapid  increase  in  interest  in  visualization  in  both  scientific  and  aviation 
contexts  has  led  many  to  search  for  “the  ultimate  form”  of  visualization  medium. 
Classical  human  factors  rhetoric  teaches  that  a  well  designed  computer  interface 
may  provide  the  user  with  an  effortless  and  eflScient  flow  of  communication  to  and 
fi’om  the  specific  device  with  which  she  or  he  interacts.  While  some  3D  computer 
displays  have  been  used  to  enhance  the  limited  capabilities  of  the  human 
information  processing  system,  in  aircraft  separation  tasks  (Ellis,  McGreevy,  & 
Hitchcock,  1987)  and  in  data  visualization  (Wickens,  Merwin,  &.  Lin  1994), 
others  have  interpreted  “well  designed”  to  necessarily  mean  advanced  technology 
(Rheingold,  1991)  and  most  recently  Virtual  Reality  (VR)  (Barfield  &  Furness, 
1995;  Durlach  &  Mavor,  1995).  This  has  led  to  the  debatable  assumption  that 
advanced  (rather  than  properly  integrated)  technology  invariably  holds  the  key  to 
successful  application.  Therefore,  Virtual  Reality’s  early  notoriety  may  have  come 
at  the  cost  of  it  being  significantly  over-sold.  VR  may  become  a  victim  of  its  own 
good  fortune  as  it  leads  to  the  use  of  features  which  may  outstrip  human 
performance  when  used  improperly. 

The  goal  for  the  research  reported  here  will  be  to  examine  certain  features 
of  virtual  reality  in  the  task  of  data  visualization.  To  do  this  we  will  examine  the 
sub-tasks  involved  in  scientific  visualization;  namely  database  navigation  and 
support  of  correct  judgments  (local  &  global)  about  the  data.  However,  before  we 
can  consider  the  possible  costs  and  benefits  of  VR  as  a  visualization  tool,  we  first 
will  look  at  the  particular  aspects  which  together  comprise  the  technology  of  VR. 

Introduction  to  Virtual  Reality 

The  technology  known  as  virtual  reality  (VR)  or  Virtual  Environment 
displays  (VEDs)  is  classified  by  Ellis  (1994)  as  interactive,  head  referenced 
computer  displays  that  give  the  users  the  illusion  of  displacem.ent  to  another 
location  (p.  17).  The  potential  for  VEDs  lie  in  their  ability  to  provide  a  new 
communication  medium  that  might  prove  to  be  cheaper,  more  intuitive,  and  more 
eflScient  than  former  interface  technologies.  These  features  should  evolve  fi-om  the 
proper  application  of  three  types  of  hardware:  (a)  sensors,  such  as  head  position 
sensors,  to  detect  the  operator’s  body  movements,  (b)  effectors,  such  as  a 
stereoscopic  display,  to  simulate  the  operator’s  senses,  and  (c)  special  purpose 
hardware  that  links  the  sensors  and  effectors  to  produce  sensory  experiences 
resembling  those  in  a  physical  environment.  Specifically  these  links  will  produce  a 
sense  of  “virtual  presence”  (p.  18). 
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Features  of  VR 


Wickens  &  Baker  (1995)  note  that  VR  is  not  a  unified  phenomenon.  It  can 
be  broken  down  into  five  separate  analyzable  features,  each  of  which  may  have 
different  effects  on  performance  as  they  are  individually  added  or  removed. 
Wickens  and  Baker  define  the  five  elements  of  reality  as  3  dimensionality,  motion, 
interaction,  an  egocentric  frame  of  reference,  and  multimodal  interaction. 

Dimensionality.  This  feature  can  be  thought  of  as  one  which  contrasts  3D 
(perspective  and  /or  stereoscopic)  viewing  with  2D  planar  viewing.  Wickens, 
Todd,  and  Seidler  (1989)  state  that  emerging  technology  in  the  areas  of  computer 
hardware  (and  software)  and  stereoscopic  imaging  technology,  has  led  to  displays 
that  represent  objects  more  naturally.  That  is,  the  displays  more  closely  resemble 
the  domain  of  objects  and  events  they  are  to  represent.  This  greater  naturalism 
was  thought  to  provide  the  viewer  with  a  better  understanding  of  the  depicted 
structure  or  environment.  For  example  a  3D  geographic  representation  is  more 
realistic  than  a  2D  contour  map.  Furthermore,  when  an  environment  is  examined 
from  a  distance  of  less  than  three  to  four  meters,  stereoscopic  viewing  can  be  used 
to  produce  a  powerful  depth  cue.  Therefore  a  3D  map  with  properly  incorporated 
stereoscopic  properties  might  be  considered  to  be  the  most  realistic  of  all  three 
maps.  VR  technology  typically  incorporates  both  elements  of  the  naturalistic 
perspective,  namely  3D  perspective  with  stereoscopic  viewing. 

Motion.  The  motion  feature  contrasts  static  vs.  dynamic  displays.  The 
environment  in  which  humans  evolved  is  very  dynamic,  therefore  a  moving 
representation  of  the  world  is  more  “real”  than  is  a  series  of  static  “snap  shots.” 

As  with  the  issue  of  dimensionality,  advances  in  computer  technology  have 
increased  the  speed  of  computer  graphic  capabilities,  thus  increasing  the  ability  of  a 
display  to  accurately  represent  a  dynamic  world.  VR  relies  heavily  on  motion  in  its 
applications,  and  has  (despite  recent  advances)  been  severely  limited  in  its  abilities 
to  provide  rapid  “real  time”  motion  simulating  the  visual  experience  of  a  user 
moving  through  an  environment.  (Smets  &  Overbeeke  1995). 

Interactivity.  This  feature  characterizes  how  the  user  interacts  with  the 
environment.  VR  interaction  is  characterized  by  an  interactive  closed  loop  mode 
(user  centered)  rather  than  open  loop  mode  (limited  or  no  feedback).  A  more 
realistic  closed  loop  mode  is  one  in  which  the  learner  has  control  over  what  aspect 
of  the  virtual  environment  is  viewed  or  visited.  In  this  sense  the  user  would  not  be 
relegated  to  the  role  of  passive  observer,  but  would  rather  be  an  active  explorer  or 
navigator  in  the  environment.  VR  incorporates  a  user  centered  mode  of 
interaction.  This  allows  the  user  to  act  on  or  explore  the  environment  as  she  or  he 
sees  fit. 


Frame  of  reference.  This  aspect  represents  the  frame  of  reference  (FOR) 
with  which  the  user  “sees”  or  observers  the  environment.  Two  of  the  possibilities 
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are  an  inside-out  (egocentric  or  immersed)  and  an  outside-in  FOR  (exocentric  or 
non-immersed).  The  immersed  perspective  is  more  realistic  because  it  incorporates 
a  vantage  point  from  the  view  point  of  the  user  (that  is  the  device  in  the 
environment  being  controlled  by  the  user).  For  example,  an  exploration  vehicle 
could  be  controlled  with  an  immersed  display  which  would  have  a  perspective 
similar  to  that  taken  from  a  camera  attached  to  the  front  of  the  vehicle.  If  a  non- 
immersed  display  was  used,  the  perspective  would  resemble  one  taken  from  a 
camera  stationed  above  or  behind  the  environment.  The  camera’s  view  would 
incorporate  the  entire  environment  (or  a  much  larger  portion  than  is  available  in 
the  immersed  display).  Therefore,  not  only  would  the  objects  in  the  environment 
be  visible  but  the  representation  of  the  user’s  exploration  vehicle  would  be  visible 
as  well.  The  exocentric  perspective  could  be  from  a  camera  fixed  in  a  single 
direction  (i.e.,  fixed  north-up)  or  from  a  camera  mounted  behind  the  user  (i.e.,  on  a 
tether),  following  behind  the  vehicle’s  track.  VR  uses  the  immersed  perspective  to 
represent  the  environment.  Thus  operators  of  VR  technology  experience  the  frame 
of  reference  as  if  they  were  actually  inside  the  environment. 

Multimodal  interaction.  There  are  a  great  variety  of  input  devices  available 
to  manipulate  the  virtual  environment.  Some  of  the  possibilities  include  speech 
recognition  and  gesture  sensation,  whether  captured  by  camera  or  sensed  through 
a  “data  glove”  (Zimmerman  &  Lanier,  1987).  Equally,  there  are  a  number  of 
feedback  sources  available  to  enhance  the  user’s  experience  of  the  virtual 
environment.  Body  motion  through  an  virtual  environment  can  generate 
proprioceptive  or  kinesthetic  feedback  to  the  users.  Tactile  feedback  is  available 
to  provide  feedback  when  users  encounter  “objects”  of  varying  textures  within  the 
virtual  environment  (Minsky  &  Ouh-Young,  1990).  Force  feedback  can  be 
delivered  through  a  joystick  or  other  hand  grips  (Brooks  &  Ouh-Young,  1990). 
Auditory  feedback  can  be  generated  through  the  use  of  3D  localized  sound 
(Begault  &  Wenzel,  1992),  and  feedback  from  head  position  can  alter  the  view  of 
the  environment  that  is  seen.  These  interaction  and  feedback  techniques  are  more 
similar  to  the  way  we  interact  with  our  normal  world  than  are  the  traditional  input 
devices  of  keyboards  and  mice. 

Mediating  Factors  on  VR  Aspects 

Wickens  and  Baker  (1995)  note  that  some  of  these  five  elements  are  not 
independent  and  will  often  interact  with  one  another.  For  example,  performance 
differences  due  to  changes  in  frame  of  reference  will  definitely  depend  on  whether 
the  display  is  dynamic  or  fixed.  Each  of  the  previously  mentioned  components  will 
produce  not  only  its  own  effect  but  also  may  interact  with  other  components. 

Furthermore,  the  influence  of  a  feature  may  be  very  much  task  dependent. 
Therefore,  factors  that  are  found  to  enhance  performance  in  one  task  may  be  found 
to  hinder  performance  in  another  task.  As  one  example,  Wickens  and  Prevett 
(1995)  examined  the  effects  of  varying  the  level  of  display  egocentrism  (immersed) 
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and  exocentrism  (non-immersed)  on  a  navigational  tracking  task  and  on  tasks  that 
assessed  the  user’s  knowledge  of  object  location  within  the  environment.  They 
found  that  an  immersed  display  produced  better  vertical  and  horizontal  navigation 
than  the  non-immersed  and  planar  displays  in  the  navigation  task.  However,  when 
analyzing  location  awareness  performance,  they  found  that  the  non-immersed 
displays  supported  performance  that  was  substantially  more  accurate  than  that 
supported  by  the  immersed  displays. 

Based  on  these  mediating  factors,  Wickens  and  Baker  (1995)  suggest  an 
analysis  be  carried  out  comparing  the  possible  benefits  of  adding  these  components 
of  reality,  with  their  potential  costs.  Most  of  these  features  will  necessitate 
significant  financial  requirements,  increases  in  system  complexity,  unreliability, 
and  maintenance  problems.  In  terms  of  possible  benefits,  one  of  the  purposes  of 
this  report  is  to  establish  whether  adding  a  particular  feature  of  reality  has  any 
benefit  to  performance  at  all.  For  the  purposes  of  this  study  we  will  examine  the 
particular  benefits  of  three  aspects  of  VR,  namely  Dimensionality  (whether  a  3D 
data  set  should  be  presented  as  a  3D  volume  or  2D  co-planar  displays), 
stereoscopic  augmentation,  and  fi'ame  of  reference  (Immersion  vs.  fixed  view 
exocentrism).  These  aspects  will  be  examined  across  four  different  conditions  as 
they  influence  a  set  of  component  tasks  frequently  employed  in  scientific 
visualization. 

VR  Applications  Examined  in  the  Present  Study 

Much  of  the  interest  in  VR  has  centered  on  its  possibilities  to  enhance 
scientific  visualization  by  facilitating  data  navigation  and  supporting  accurate  data 
judgments.  It  is  possible  to  imagine  a  scientist  who  would  use  VR  as  a  tool  to 
examine  a  particular  multidimensional  (greater  than  2  dimensions)  environment. 

For  example  consider  the  scientist,  analyzing  the  distribution  of  toxic 
concentrations  based  upon  geographic  location  (X  &  Z  dimension)  as  well  as 
depth  (Y  dimension).  The  scientist  may  find  it  necessary  to  “zoom  in”  and 
examine  a  particular  concentration  as  well  as  “zoom  out”  to  gain  a  more  holistic 
understanding  of  the  distribution  of  toxic  concentrations  in  terms  of  their  overall 
location,  patterns,  shape,  etc. 

For  the  purposes  of  this  experiment  we  will  consider  scientific  visualization 
as  a  dual  component  process:  Database  navigation  and  Judgment  support. 

Database  navigation  can  be  fiirther  divided  into  two  subtasks:  Search  and  travel. 
The  search  subtask  is  triggered  by  the  user’s  need  to  locate  an  object  of  interest  in 
the  virtual  environment.  This  could  correspond  to  our  scientist’s  desire  to  locate  a 
particular  concentration  to  examine.  It  could  also  be  applied  to  locating  a  potential 
aircraft  conflict  in  a  virtual  ATC  environment,  an  object  representing  a  reference  in 
a  data  base  (Newby,  1992),  an  item  in  a  menu  (Jacoby  &  Ellis,  1992),  a  fact  in  a 
hypertext  space  (Golovchinsky  &  Chignell,  1993),  a  particular  data  point  in  a 
scientific  data  base  (Wickens,  Merwin,  &  Lin,  1994)  or  a  tumor  in  a  representation 
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of  body  tissue  (Abarbanel,  FriedhofiF,  Langridge,  Pearlman,  &  Star  1993).  The 
common  element  between  these  studies  is  that  all  of  these  tasks  require  a  visual 
search  of  an  unknown  environment  for  some  designated  “target.”  Once  the  object 
is  discovered  visually,  there  may  also  be  a  need  to  orient  the  VR  device  on  a 
course  toward  the  object,  so  that  travel  may  take  place. 

The  travel  subtask  is  triggered  by  the  user’s  need  to  navigate  quickly  and 
accurately  throughout  the  virtual  environment  (VE)  to  attain  a  closer  (zoom  in) 
view  of  a  certain  object.  Because  VEs  are  not  necessarily  tied  to  the  natural 
physical  laws,  travel  through  VEs  can  be  instantaneous.  While  this  may  increase 
the  actual  speed  and  flexibility  of  the  navigation  process,  it  may  have  severe 
consequences  in  terms  of  situational  awareness.  Warren  and  Wertheim  (1990) 
suggest  that  locomotion  through  a  virtual  environment  should  capitalize  as  much 
as  possible  on  the  user’s  own  natural  navigation  experiences.  Based  on  this 
suggestion  we  would  not  desire  to  have  instantaneous  motion  within  our 
environment  because  operators  are  not  used  to  navigating  that  way.  Furthermore, 
Wickens  and  Baker  (1995)  suggest  that  virtual  travel  should  represent  a 
compromise  between  two  features;  speed  and  situational  awareness,  the  latter  of 
which  may  be  lost  with  rapid  travel  speeds.  Loss  of  situational  awareness  is  a 
major  concern  in  the  aviation  community  (Billings,  1991;  Wiener,  1989),  and  it  is 
also  a  potential  concern  in  terms  of  how  it  degrades  scientific  visualization 
performance.  Therefore,  for  purposes  of  this  experiment,  travel  throughout  the 
VE  was  not  designed  to  be  instantaneous. 

Regarding  the  judgment  task,  this  report  considered  two  subtasks:  Local 
(relative)  judgments  and  Global  (integrative)  judgments.  Local  judgments  can  be 
characterized  by  how  well  the  displayed  information  facilitates  the  user’s 
evaluation  of  an  object’s  position.  This  position  can  also  be  characterized  either  as 
relative  (in  relation  to  other  objects)  or  as  absolute  (i.e.,  measured  along  a  single 
scale),  such  as  individually  measuring  heat,  volume,  depth,  etc.).  Due  to  the  large 
number  of  possible  absolute  scales  and  the  specific  nature  of  these  scales,  this 
experiment  focused  on  only  the  relative  judgment  support. 

Global  judgments  can  be  thought  of  as  measures  of  the  viewer’s  ability  to 
grasp  the  environment  as  a  whole.  For  example  can  the  viewer  meaningfully 
understand  the  distribution,  shape  or  size  of  the  environment  that  she  or  he  just 
encountered?  This  is  the  goal  of  many  scientific  visualization  applications  of  VR 
and  therefore  will  be  examined  in  this  report.  Specifically,  we  will  measure  global 
judgment  by  the  viewer’s  accuracy  in  describing  the  distribution  or  shape  of 
objects  within  the  VE. 
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REVIEW  OF  THE  LITERATURE 


In  the  following  pages,  we  now  review  the  literature  which  has  examined 
the  three  features  investigated  here  on  the  performance  of  the  scientific 
visualization  subtasks  described  in  the  Introduction. 

Dimensionality  and/or  Stereo  (D&SI  Effects  on  the  Visualization  Subtasks 

In  the  following  section,  we  will  focus  on  how  3D  and  stereo  viewing 
conditions  were  found  to  affect  the  visualization  subtasks  addressed  in  this 
experiment:  Search,  Travel,  Local  Judgment  Support,  &  Global  Judgment 
Support. 

D&S  effects  on  search.  While  little  experimental  data  exist  on  how 
dimensionality  and  stereoscopic  viewing  affect  the  search  task,  an  intuitive 
example  will  provide  some  guidance.  First,  we  must  clarify  that  the  search  task  is 
actually  composed  of  two  separate  subtasks;  target  location  and  icon  orientation. 
Target  location  can  be  defined  as  the  time  and/or  effort  required  by  the  subject  to 
find  the  target  (i.e.,  flashing  box,  colored  symbol,  etc.).  The  salience  of  the  target, 
relative  to  the  background,  will  be  the  biggest  determining  factor  in  this  “finding” 
task.  Therefore  if  all  factors  affecting  salience  (i.e.,  target  luminance,  viewing 
distance,  background  noise,  etc.)  are  held  constant  on  all  displays,  then  we  should 
expect  little  difference  between  the  2D  and  the  3D  formats. 

The  icon  orientation  subtask  requires  the  user  to  position  her  or  his  icon  on 
a  trajectory  that  will  intercept  the  target,  which  is  already  in  sight.  To  properly 
align  the  icon,  the  subject  must  have  a  clear  picture  of  both  “where  they  are  now” 
and  “where  they  need  to  go.”  Intuitively,  the  integrated  3D  display  should  show 
an  advantage  for  this  task.  With  all  three  dimensions  integrated  into  one  display, 
the  user  only  has  to  attend  to  the  one  display.  The  planar  2D  displays  will  require 
a  division  of  visual  attention  between  two  panels  (i.e.,  scanning).  Added  to  this, 
the  3D  display  removes  the  need  for  mental  integration  between  the  two  separated 
screens  of  the  2D  display. 

Ellis,  McGreevy,  and  Hitchcock  (1987)  compared  plan  views  (top  down 
2D  representation)  vs.  perspective  views  (3D  representation)  of  air  traffic  displays 
in  an  air  traffic  avoidance  task.  They  found  an  approximate  10%  time  reduction 
for  interpretation  of  the  perspective  display.  In  similar  work,  Bemis,  Leeds  and 
Winer  (1988)  found  3D  perspective  displays  revealed  a  substantial  advantage  over 
2D  planar  displays  in  an  airborne  threat  identification  (i.e.,  search)  and  interception 
coordination  task. 

Liang,  Wickens,  and  Olmos  (1995)  explored  the  effect  of  display 
dimensionality  on  tasks  involving  the  estimation  of  an  absolute  bearing  toward 
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terrain  features;  a  task  similar  to  “icon  orientation.”  The  results  showed  that  3D 
displays  supported  significantly  faster  response  times  than  2D  displays  for  absolute 
bearing  judgments.  These  studies  cite  the  reduction  of  scan  time  as  significant 
contributors  to  the  success  of  the  3D  displays  relative  to  the  2D  displays. 
Therefore,  it  seems  the  3D  display  supports  better  search  performance  by 
eliminating  the  scan  between  displays. 

If  3D  viewing  is  advantageous  for  some  aspects  of  the  search  task,  it  might 
be  anticipated  that  enhancing  the  sense  of  3D  by  means  of  stereoscopic  viewing 
would  provide  further  performance  enhancement.  Several  studies  have  shown  a 
stereopsis  benefit  for  the  performance  of  such  tasks  as  simulated  rotorcraft 
performance  (Parish  &  Williams;  1990)  and  neurosurgery  (Sollenberger  & 
Milgram,  1989).  Specific  to  a  search  task,  Zenyuh,  Reising,  Walchli,  and  Biers 
(1988)  compared  the  relative  abilities  of  a  stereoscopic  3D  display  versus  a 
conventional  2D  display  to  support  the  task  of  searching  spatial  quadrants  for 
targets  in  a  simulated  air  to  air  combat  display.  Zenyuh,  et  al.  found  greater 
accuracy  for  the  3D  stereoscopic  display  relative  to  the  2D  display.  Response  time 
was  not  significantly  different  between  the  two  display  formats.  Way  (1988) 
investigated  whether  stereo  would  enhance  performance  in  the  detection  of 
malfunctions  of  a  simulated  engine  system.  Way  found  that  stereoscopic  viewing 
used  with  a  color  code  resulted  in  fewer  errors  than  a  3D  display  with  the  color 
code  alone,  although  it  did  not  reduce  response  time.  However  when  stereo  was 
used  alone,  a  significant  increase  in  response  time  was  found  relative  to  the 
conditions  featuring  only  color  code  and  color  code  combined  with  stereo.  Hence 
we  might  conclude  that  the  benefits  of  stereo  to  search  are  ambivalent,  and 
probably  depend  a  good  deal  on  what  is  being  searched  for  and  the  relative 
salience  of  other  cues  in  discriminating  the  target  from  the  background. 

D&S  effects  on  travel.  One  of  the  major  selling  points  of  VR  technology  is 
that  it  allows  for  a  deeper  exploration  of  the  database  by  allowing  the  viewer  to 
“zoom  in”  toward  a  particular  point  of  data.  This  exploration  is  assumed  to 
facilitate  a  better  appreciation  of  the  relation  between  variables  than  can  be 
achieved  by  separated  2D  displays.  However,  this  exploration  requires  the  viewer 
to  have  the  ability  to  travel  through  the  database.  Therefore,  research  discussing 
travel  and  its  relation  to  dimensionality  and  stereoscopic  viewing  is  important  to 
this  study. 

When  defining  the  traveling  subtask  for  this  experiment,  we  must  consider 
both  the  cognitive  elements  (How  well  can  viewers  avoid  getting  lost  in  the  virtual 
space?)  and  the  perceptual-motor  (i.e.,  tracking)  elements  (How  well  can  viewers 
control  the  viewpoint  in  virtual  space?).  The  specific  methodology  of  the 
experiment  guards  against  the  cognitive  concerns;  making  motor  control  our  chief 
concern.  Haskell  and  Wickens  (1993)  found  that  3D  displays  led  to  superior 
control  in  the  lateral  and  vertical  axes  along  the  flight  path  during  a  landing 
approach  in  a  simulated  aircraft,  an  effect  replicated  by  Wickens  and  Prevett 
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(1995).  However,  the  2D  display  was  shown  to  have  superior  performance  in 
terms  of  controlling  airspeed.  The  advantage  for  3D  was  attributed  to  the  reduced 
scanning  with  the  integrative  displays,  while  the  cost  of  the  3D  display  for  speed 
control  was  attributed  to  the  fact  that  3d  displays  are  typically  ambiguous  in 
representing  distance  along  the  viewing  axis  (line  of  sight  ambiguity). 

Wickens,  Liang,  Prevett,  and  Olmos  (1994)  also  compared  2D  and  3D 
displays  to  examine  their  effects  on  navigation  performance.  The  results  indicated 
no  significant  difference  between  the  3D  and  2D  perspectives  in  terms  of  lateral 
tracking  performance.  Furthermore,  they  noted  some  3D  costs  in  vertical  tracking 
performance.  However  these  costs  were  small,  and  the  results  showed  a  trend 
toward  3D  cost  reduction  with  practice. 

Wickens  and  Prevett  (1995),  using  a  similar  paradigm  as  Haskell  and 
Wickens  (1993),  found  that  an  exocentric  (non-immersed)  3D  display  was  no 
better  than  the  2D  display  in  navigation  tasks.  As  did  Haskell  and  Wickens,  they 
concluded  that  the  exocentric  view  produces  ambiguity  along  the  line  of  sight. 

The  augmentation  provided  by  adding  stereo  to  3D  displays  has  also  been 
examined  in  some  3D  navigational  tasks.  The  field  of  telerobotics  provides  a 
natural  testing  ground  for  examining  the  possible  benefits  of  using  stereo.  Most  of 
the  actual  environments  encountered  by  teleoperators  are  3D  volumes,  which 
require  efficient  user  feedback  and  a  spatially  orientated  interface  to  promote 
successfiil  exploration.  Pepper,  Smith,  and  Cole  (1981)  and  Pepper,  Cole,  Spain, 
and  Sigurdson  (1983),  found  that  stereoscopic  viewing  provided  better  tracking 
performance,  but  that  the  advantage  was  mediated  by  visibility,  task  type,  and 
learning  factors.  Stereoscopic  viewing  had  a  greater  advantage  over  monoscopic 
viewing  when  scene  complexity  was  increased  and  object  visibility  was  decreased. 
In  a  related  study.  Cole,  Merrit,  Coleman,  and  Dcehara  (1991)  required  subjects  to 
guide  a  remote  operator  through  a  wire  maze.  When  subjects  used  a  stereoscopic 
display  they  completed  the  task  faster  and  with  higher  accuracy  than  with  a 
monoscopic  display.  Furthermore,  Cole  et  al.  found  that  subjects  explored  the 
wire  maze  in  a  slow  trial  and  error  fashion  when  using  the  monoscopic  display,  but 
not  when  using  the  stereoscopic  display.  This  suggests  that  the  stereoscopic 
display  gave  the  subjects  a  better  idea  of  the  spatial  characteristics  of  the  maze 
than  the  monoscopic  display. 

Drasic  (1991)  found  similar  stereoscopic  benefits  in  reducing  response 
latencies  and  execution  time  in  a  teleoperation  task.  Furthermore,  Drasic  found 
that  stereoscopic  viewing  reduced  both  the  time  required  for  training  and  the  error 
rates. 


Stereoscopic  displays  have  also  been  examined  in  non-teleoperation  tasks. 
For  example,  monoscopic  and  stereoscopic  displays  have  been  compared  with 
regard  to  how  they  affected  performance  on  a  three-axis  manual  tracking  tasks 
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(Ellis,  Kim,  Tyler,  McGreevy,  &  Stark,  1985;  Kim,  Ellis,  Tyler,  Hannaford,  & 
Stark,  1987).  These  studies  found  that  stereoscopic  3D  displays  supported  smaller 
tracking  errors  than  did  their  3D  monoscopic  counterparts.  Furthermore, 
Sollenberger  and  Milgram  (1993)  replicated  these  studies  and  found  that 
stereoscopic  displays  produced  substantial  benefits  in  performance  in  a  3D  path¬ 
tracing  task.  Nataupsky  and  Crittenden  (1988)  however,  observed  few  benefits  of 
stereo  in  flight  path  tracking  when  pilots  viewed  a  3D  “highway  in  the  sky”,  given 
the  presence  of  other  rich  depth  cues. 

Summarizing  these  experiments,  it  seems  that  stereoscopic  displays 
generally  support  better  navigation  performance.  Much  like  the  superior 
navigation  performance  of  3D  over  2D,  we  can  assume  that  stereo  superiority  is 
due  to  its  ability  to  present  a  heightened  sense  of  perspective  parallel  to  the 
viewing  axis,  often  needed  by  the  user  and  often  poorly  supported  by  other  depth 
cues. 


D&S  effects  on  local  judgments.  Several  studies  have  examined  whether 
the  use  of  a  perspective  displays  may  support  more  accurate  judgments  of  spatial 
information  than  would  be  found  using  planar  (2D)  displays.  Burnett  and  Barfield 
(1991)  found  that  a  3D  display  of  a  simulated  air  traffic  environment  resulted  in 
more  accurate  recall  of  an  aircraft’s  position  compared  with  performance  using  a 
planar  display.  While  these  results  may  seem  encouraging  for  supporting  the  idea 
that  3D  will  led  to  better  local  judgment  performance,  other  studies  have  arrived  at 
different  conclusions. 

Wickens  and  Todd  (1990)  found  that  on  tasks  requiring  the  relative 
judgment  of  position  across  a  single  dimension,  no  significant  difference  existed 
between  a  2D  (bar  graph)  display  and  a  3D  integrated  display.  However,  they 
found  a  shorter  response  latency  for  the  2D  display.  Wickens  and  May  (1994) 
contrasted  how  2D  vs.  3D  displays  would  affect  the  subjects’  abilities  to  judge 
aircraft  trajectory  relative  to  mountainous  terrain.  Wickens  and  May  found  that  air 
traffic  controller  subjects  were  less  accurate  with  the  3D  display  than  with  the  2D 
display  in  judging  complex  3D  trajectories.  Pilot  subjects  in  contrast  showed  no 
significant  difference  between  display  types. 

Wickens,  Liang,  Prevett,  and  Olmos  (1994)  found  that  the  pilot’s  ability  to 
indicate  the  relative  vertical  position  of  a  given  navigational  feature  was  superior  in 
the  2D  format,  while  the  3D  display  format  supported  faster  response  times. 
Wickens  and  Prevett  (1995)  used  a  similar  paradigm  and  found  that  there  was  no 
significant  difference  in  response  time  to  local  judgment  questions.  However, 
there  was  a  significant  advantage  in  accuracy  for  the  2D  displays. 

Many  of  the  these  studies  state  that  relative  accuracy  may  be  degraded  in 
3D  displays  due  to  line-of-sight  ambiguities.  Furthermore,  they  propose  that  the 
shortened  response  latency  of  the  3D  displays  was  due  to  the  elimination  of 
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scanning  between  two  planar  displays.  According  to  these  studies,  there  may  be  a 
disadvantage  to  using  3D  displays  for  local  judgments  inside  complex  3D 
environments. 

However,  when  2D  and  3D  displays  were  compared  in  a  scientific 
visualization  task,  the  results  were  somewhat  different.  Wickens,  Merwin,  and  Lin 
(1994)  varied  display  dimensionality  to  examine  possible  techniques  to  assist 
scientists  in  evaluating  multidimensional  data.  Subjects  viewed  a  series  of  complex 
3D  data  sets,  fi-om  which  a  set  of  discrete  points  or  observations  were  sampled.  In 
one  experiment,  subjects  viewed  complex  data  sets  in  both  3D  and  2D  displays  and 
answered  “on-line”  questions  about  the  relative  values  of  displayed  information. 
They  found  that  2D  displays  were  responded  to  more  slowly  than  3D  displays. 
Furthermore,  they  found  that  on  questions  requiring  integration  of  information,  the 
3D  displays  were  responded  to  more  accurately  than  the  2D  displays.  Based  upon 
this  work  it  seems  that  3D  displays  should  be  the  recommendation  to  those 
interested  in  making  local  judgments  within  scientific  visualization  contexts. 
However,  other  viewing  parameters  known  to  heighten  the  realism  of  the  display, 
notably  stereoscopic  viewing,  must  also  be  considered. 

In  this  regard,  Yeh  and  Silverstein  (1992)  found  that  the  addition  of 
binocular  disparity  to  a  perspective  display  led  to  faster  and  more  accurate 
judgments  of  altitude  and  depth  of  objects  in  a  3D  volume  than  was  accomplished 
with  a  perspective  display  alone.  In  a  similar  study,  Barfield  and  Rosenberg  (1995) 
found  that  stereoscopic  displays  resulted  in  more  accurate  estimates  of  elevation 
angle  between  two  subjects,  while  no  benefit  was  found  in  estimating  azimuth. 
Wickens,  Merwin,  and  Lin  (1994)  also  varied  the  presence  of  stereo  in  their 
experiment.  Stereo  views  of  the  3D  display  were  found  to  support  better  local 
judgment  performance  above  than  that  found  in  monoscopic  3D  displays. 

Based  upon  the  results  of  these  experiments,  we  can  see  that  the  use  of 
stereoscopic  viewing  does  increase  the  accuracy  of  relative  judgments.  These 
advantages  suggest  that  3D  displays  combined  with  stereo  will  achieve  superior 
accuracy  to  2D  planar  displays. 

D&S  effects  on  global  judgments.  Much  of  the  previous  research  on  global 
judgments  regarding  the  structure  of  the  database  comes  from  examination  of 
dimensionality  and  its  effects  on  global  situational  awareness  in  aviation.  Rate  and 
Wickens  (1993)  examined  global  judgment  support,  while  pilots  flew  a  simulated 
approach  task  and  found  that  assessments  of  the  pilot’s  ability  to  spatially  localize 
terrain  features  was  better  with  the  use  of  2D  displays.  Furthermore,  the  results 
indicated  that  pilots  showed  no  time  advantage  for  the  3D  display. 

Wickens,  Liang,  Prevett,  and  Olmos  (1994)  used  two  measures  to  assess 
dimensionality  effects  on  global  judgments;  a  fi-ozen  screen  test  and  a  map 
reconstruction  test.  The  fi’ozen  screen  test  halted  the  task  temporarily  and  pilots 
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were  asked  questions  referring  to  the  global  aspects  of  the  environment  (i.e.,  the 
shape  of  certain  terrain  features  or  altitude  and  heading  changes  necessary  to 
intercept  an  object).  The  map  reconstruction  task,  given  after  completion  of  the 
final  landing  task,  required  the  pilots  to  draw^  as  many  terrain  features  as  they  could 
remember  encountering  and  to  indicate  the  direction  of  north.  They  found  that  on 
the  fi'ozen  screen  tests,  the  2D  displays  supported  faster  response  times  fro  the 
pilot  to  detect  a  collision  given  a  specific  altitude  and  heading  change. 

Furthermore,  the  2D  displays  supported  more  accurate  placement  of  terrain 
features  for  the  map  drawing  task  than  did  the  perspective  3D  display.  As  with  the 
other  “aviation  studies”  (Wickens  &  May,  1994;  Wickens  &,  Prevett,  1995), 
Wickens  et  al.  speculated  that  3D  displays’  accuracy  may  be  degraded  due  to  line- 
of-sight  ambiguities. 

In  their  examination  of  visualization  of  a  3D  database,  Wickens,  Merwin, 
and  Lin  (1994)  as  noted  above,  found  that  “on  line”  questions  requiring  the  most 
integration  (global  Judgments)  were  better  supported  by  3D  displays  rather  than 
2D  displays.  Further  more  they  tested  subjects  for  their  retention  of  the  surface 
characteristics,  after  a  number  of  samples  were  viewed  from  each  surface.  This 
was  assessed  by  a  vmtten  post-test  which  asked  them  to  provide  detailed 
knowledge  of  the  data  base  (i.e.,  shape  formed  by  the  data  points  and  values  of  the 
variables  at  specific  regions  of  the  data  space).  The  first  experiment  compared  3D 
vs.  2D  displays  on  the  post-test  and  revealed  no  significant  differences  in  global 
judgments.  In  a  second  experiment,  Wickens,  Merwin  and  Lin  (1994)  found  that 
stereoscopic  viewing  facilitated  better  performance  on  the  global  “on  line” 
questions  than  3D  monoscopic  viewing,  but  that  stereo  had  no  benefit  on  the  long 
term  memory  for  the  structure  of  the  database. 

While  most  of  the  studies  suggest  that  there  is  no  clear  global  judgment 
advantage  for  one  type  of  dimensionality  (3D  or  2D),  the  work  of  Wickens, 

Merwin  and  Lin  (1994),  suggests  otherwise.  Furthermore,  this  latter  work  is  the 
most  related  to  the  present  study  and  the  field  of  scientific  visualization. 

Therefore,  we  speculate  that  3D  displays  should  surpass  2D  displays  in  terms  of 
supporting  global  judgments  of  the  structure  of  the  data.  Furthermore,  as  noted  in 
Wickens,  Merwin  ,and  Lin,  stereoscopic  viewing  should  not  improve  global 
judgments. 

Frame  of  Reference  (FORI  Effects  on  the  Visualization  Subtasks 

Previous  studies  do  not  support  a  single  “best”  display  with  respect  to 
dimensionality  and  stereo.  Despite  this  inconclusiveness,  it  has  been  suggested 
that  because  humans  interact  with  their  world  in  terms  of  three  dimensions,  the 
optimal  interface  should  resemble  that  environment  as  closely  as  possible.  In  terms 
of  VR,  this  would  mean  the  immersive  perspective  of  VR,  representing  the  highest 
level  of  realism  to  the  subject,  should  allow  for  the  most  efficient  interface. 


14 


However,  we  must  remember  that  “optimal  performance”  and  “optimal  reality”  are 
not  necessarily  the  same  and  it  is  the  former  that  is  the  goal  of  these  interfaces. 

Therefore  in  the  following  section,  we  will  examine  research  addressing  the 
effects  of  modifying  the  frame  of  reference  (FOR)  in  the  4  visualization  subtasks 
addressed  in  this  experiment;  Search,  Travel,  Local  Judgment  Support,  &  Global 
Judgment  Support. 

FOR  and  search.  As  is  common  with  VR  research,  there  is  a  shortage  of 
empirical  studies  which  addresses  the  issue  of  frame  of  reference  and  its  effect  on 
search  performance.  As  stated  earlier,  the  search  task  actually  is  comprised  of  two 
subtasks;  target  location  and  icon  orientation.  In  our  comparison  of  2D  vs.  3D 
displays,  the  two  sub-tasks  could  be  addressed  separately.  However  when 
considering  frame  of  reference,  it  will  be  impossible  to  separate  the  two  tasks  at 
certain  times.  An  example  of  this  can  be  found  when  an  egocentric  (immersed) 
FOR  is  used.  Due  to  the  fact  that  the  viewing  perspective  is  depicted  from  the 
front  of  the  icon  itself,  orienting  the  icon  toward  the  target  is  the  only  way  the  user 
can  locate  the  next  target  hence  location  and  orientation  are  accomplished  in 
parallel.  On  the  other  hand,  the  exocentric  (non-immersed)  FOR  will  require  that 
the  two  subtasks  be  addressed  separately.  The  detached  “birds-eye”  viewpoint  will 
allow  the  viewer  to  survey  the  entire  database  visually,  without  “actively”  locating 
the  next  target  via  manual  manipulation.  The  viewer  will  locate  the  target  by 
visual  search  and  will  then  proceed  directly  with  the  manual  orientation  task. 

The  immersed  egocentric  viewpoints  are  limited  in  this  sense  by  their  “key 
hole”  views  because  the  viewpoint  will  only  allow  the  viewer  to  search  the  field  of 
view  that  she  or  he  can  see  through  the  “key  hole”  at  that  time.  This  keyhole  view 
can  be  expanded,  by  enlarging  the  geometric  field  of  view  (GFOV)  thus  increasing 
the  user’s  view  of  the  data,  (Kim,  Ellis,  Tyler,  Hannaford,  &  Starl^  1987;  Wickens 
&  Prevett,  1995).  However,  increasing  the  GFOV  may  result  in  “nonecological” 
distortions  which  could  hinder  local  judgments  (McGreevy  &  Ellis,  1986). 
Therefore,  in  the  experiment  we  reported  below  the  GFOV  was  not  expanded, 
thus  resulting  in  a  limited  key  hole  view  for  the  immersed  FOR  condition.  Based 
upon  this  line  of  thinking,  it  seems  that  the  unrestricted  field  of  view  inherent  in  the 
non-immersed  FOR  should  support  better  search  performance. 

FOR  and  travel.  Several  studies  have  examined  the  effects  that  an 
egocentric  vs.  and  exocentric  FOR  would  have  on  aircraft  flight.  In  these  studies, 
the  contrast  of  reference  frames,  has  been  varied  in  two  different  comparisons; 
immersion  vs.  exocentric,  and  rotation  vs.  fixed.  Both  comparisons  deal  with 
comparing  a  more  egocentric  vs.  a  more  exocentric  viewpoint.  The  more 
egocentric  FORs  would  include  viewing  the  environment  as  a  pilot  would 
(immersive  viewing)  or  using  a  rotating  map  that  corresponds  to  the  pilot’s  outside 
view.  The  exocentric  FORs  would  include  viewing  the  environment  as  a  viewer 
would  if  detached  from  the  vehicle  (non-immersed)  or  using  a  map  fixed  in  a 
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certain  (typically  north-up)  direction.  Rate  and  Wickens  (1993)  compared 
performance  for  pilots  on  a  landing  task  using  either  a  rotating  track-up  or  a  fixed 
north-up  displays  and  found  that  the  track-up  (egocentric)  displays  were  better  for 
lateral  flight  control.  These  results  were  replicated  by  both  Wickens,  Liang, 
Prevett,  and  Olmos  (1994)  and  by  Liang,  Olmos,  and  Wickens  (1995),  who  found 
that  more  exocentric  (fixed)  maps  produced  larger  deviations  in  both  lateral  and 
vertical  dimensions  than  did  egocentric  maps. 

Wickens  and  Prevett  (1995)  examined  the  effects  of  vaiydng  degrees  of 
exocentrism  on  tracking  performance.  Exocentricity  was  varied  by  adjusting  the 
distance  (tether  length)  which  the  viewpoint  was  positioned  away  fi’om  the  aircraft 
(i.e.,  the  degree  of  “immersion”).  The  results  revealed  that  the  fiilly  immersive 
FOR  supported  better  tracking  performance  in  both  the  vertical  and  horizontal 
axes  than  either  the  planar  or  exocentric  perspective  displays.  Using  a  similar 
paradigm,  Barfield,  Rosenberg,  and  Furness  (1995)  found  the  least  exocentric  (full 
immersive)  display  supported  the  lowest  tracking  error  and  quickest  time  to  reach 
a  target. 

Further  research  was  conducted  using  the  “Tunnel  in  the  Sky”  primary 
flight  display.  The  tunnel  in  the  sky  is  a  perspective  display  which  functions  much 
like  a  flight  director,  but  provides  a  look-ahead  capability  and  basic  ground 
reference  elements.  It  also  uses  an  immersed  (egocentric)  frame  of  reference.  The 
tunnel  in  the  sky  system  has  been  found  to  enhance  performance  relative  to  a  non- 
immersive  display  on  simulated  three  dimensional  helicopter  approaches 
(Grunwald,  Robertson,  &  Hatfield;  1981). 

Related  research  was  performed  by  McGovern  (1993)  who  studied 
accidents  involving  teleoperated  vehicles  at  the  Sandia  National  Laboratories 
(SNL).  Subjects  in  the  immersed  group  controlled  remotely  operated  vehicles 
equipped  with  direct  video  and  audio  feedback.  Subjects  in  the  non-immersed 
group  had  no  such  links  and  was  forced  to  operate  the  vehicles  from  a  fixed 
position.  The  operators  remotely  navigated  the  vehicles  through  an  off-road  track. 
Nine  of  the  ten  accidents  which  occurred  in  the  “immersive  group”  involved  a  loss 
of  control  due  to  a  miscalculation  of  the  vehicles’  dynamics  (e.g.,  hitting  a  bump  or 
turning  too  fast)  (p.  186).  The  non-immersive  group  had  a  similar  number  of 
accidents,  however  those  accidents  involved  underestimating  stopping  distances 
(hitting  a  fence),  problems  with  depth  perception  (hitting  a  tree  and  a  post ),  and 
control  reversals  (hitting  a  car).  This  suggests  that  the  non-immersive  group’s 
problems  involved  misunderstanding  the  environment  within  which  they  were 
operating.  Therefore,  non-immersive  frames  of  reference  seems  to  cause 
confusion  to  the  navigate  about  the  nature  of  the  environment  and  her  or  his  place 
inside  it. 

Summarizing  these  results,  we  find  that  egocentric  FOR,  whether  achieved 
by  map  rotation  or  by  immersion  of  the  viewpoint,  produces  better  tracking 
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performance.  Based  on  these  experiments,  we  expect  the  immersive  display  to 
support  superior  travel  performance  relative  to  both  the  non-immersed  3D  display 
and  the  planar  2D  display. 

FOR  and  local  judgments.  Several  experiments  have  examined  how  FOR 
affects  relative  judgment  performance.  Barfield  and  Kim  (1991)  varied  FOR  by 
altering  the  GFOV  and  the  tether  lengths  of  the  subjects’  viewpoint  and  found  that 
a  mid-exocentric  FOR  produced  the  lowest  azimuth  error  of  estimation  of  target 
location,  but  the  greatest  errors  in  elevation  judgment.  Using  a  similar  paradigm, 
Wickens  and  Prevett  (1995)  varied  degrees  of  exocentricity  (via  their  tether 
lengths)  and  found  no  difference  in  response  time,  but  that  increasing  exocentricity 
generally  led  to  higher  accuracy  in  relative  judgments  of  target  location.  The  trend 
for  accuracy  was  reversed  when  an  extreme  exocentric  viewpoint  was  used; 
resulting  in  lower  accuracy  than  a  mid-exocentric  point  of  view. 

Wickens,  Liang,  Prevett,  and  Olmos  (1994)  had  pilots  report  the  relative 
bearing  of  a  terrain  feature  and  judge  whether  it  was  above  or  below  the  simulated 
aircraft.  Pilots  flew  with  either  fixed  north-up  maps  (exocentric)  or  rotating  maps 
(egocentric).  There  were  no  significant  differences  between  the  two  map  types  in 
above/below  judgment  accuracy,  while  the  fixed  north-up  map  supported  more 
accurate  estimates  of  relative  bearing  and  produced  faster  reaction  times  than  the 
rotating  map.  Using  the  same  fixed  vs.  north-up  paradigm,  Liang,  Wickens  and 
Olmos  (1995)  assessed  performance  on  a  task  which  required  pilots  to  report  an 
absolute  bearing  upon  which  they  plane  would  fly  to  intercept  an  unexpected 
target.  Liang  et  al.  found  no  difference  in  response  time  or  judgment  accuracy 
between  the  two  map  display  types. 

After  reviewing  these  studies,  it  is  still  unclear  whether  increasing  the 
degree  of  exocentricity  of  the  display  will  improve  local  judgment  task 
performance.  Therefore  further  work,  such  as  that  which  we  report  in  this 
experiment,  must  be  done  to  see  how  different  FOR  will  affect  relative  judgment 
support. 

FOR  and  global  judgments.  Aretz  (1991)  examined  the  performance 
differences  between  a  track-up  map  and  a  fixed  north-up  map  on  a  map 
reconstruction  task.  The  task  required  the  subject  to  place  landmarks,  encountered 
in  flight  simulation,  in  their  appropriate  locations  after  completion  of  the  flying 
task.  The  results  indicated  that  more  landmarks  were  recalled  in  their  correct 
locations  with  the  north-up  (exocentric)  displays. 

In  a  similar  study,  Wickens  et  al.  (1994)  examined  differences  in  global 
judgments  between  north-up  vs.  track-up  maps.  Specifically  they  examined  the 
FOR  affects  in  the  same  ways  as  they  examined  the  dimensionality  affects. 
Additionally,  a  map  reconstruction  task  was  given  to  the  subjects  after  the  final 
landing  task  had  been  completed.  They  found  that  pilots  had  quicker  response 
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times  when  using  the  track-up  (immersed)  display  for  the  frozen  screen  tests. 
However  there  was  no  difference  in  accuracy  between  the  two  displays.  The 
results  of  Wickens  et  al.  replicated  the  findings  of  Aretz  (1991)  by  finding  a 
significant  advantage  for  the  north-up  (non-immersed)  display  in  the  map 
reconstruction  task. 

Further  replication  was  provided  by  a  study  by  Williams,  Hutchinson,  and 
Wickens  (1994)  who  examined  how  fixed  versus  rotating  frames  of  reference  used 
during  training  would  influence  later  recall  of  the  terrain’s  configuration.  Williams 
et  al.  categorized  participants  in  terms  of  how  they  held  the  map  during  training; 
finding  that  60%  of  the  subjects  consistently  held  the  map  north-up  and  that  28% 
rotated.  Assuring  that  the  distribution  of  map  orientation  preference  was  uniform, 
Williams  et  al.  evaluated  pilots  on  a  map  reconstruction  task  (similar  to  that  used 
by  Aretz,  1991;  Wickens  et  al.,  1994)  and  found  that  the  north-up  (exocentric) 
map  participants  had  significantly  higher  map  reconstruction  scores  that  their 
track-up  (egocentric)  map  counterparts.  It  should  be  noted,  however,  that  the 
advantage  of  north  up  versus  track  up  maps  for  terrain  recall  has  not  always  been 
found  (Liang  et  al.,  1995). 

Finally,  in  the  experiment  described  above,  Wickens  and  Prevett  (1995) 
investigated  how  varying  exocentricity  -  via  tether  length  would  affect  global 
judgments.  Using  the  same  map  reconstruction  task,  they  found  no  significant 
difference  between  the  displays  using  the  varying  degrees  of  exocentricity. 

Barfield,  Rosenberg,  and  Furness  (1995)  compared  immersed  (egocentric)  and 
non-immersed  (exocentric)  displays  and  found  that  pilots  flying  with  an  exocentric 
frame  of  reference  yielded  better  map  reconstruction  than  pilots  flying  with  an 
egocentric  frame  of  reference.  The  results  of  these  five  experiments  seem  to 
suggest  that  the  exocentric  viewpoint  tend  to  support  global  judgments  that  are  as 
good  or  better  than  the  egocentric  immersed  displays. 

Hypotheses  and  Proposed  Experiment 

The  previous  review  has  identified  a  number  of  studies  that  have  examined 
the  mfluence  of  the  three  features  of  VR  (frame  of  reference,  dimensionality,  and 
stereo),  on  the  four  subtasks  of  visualization;  however,  most  of  these  studies  have 
involved  navigation  and  judgments  in  an  aerospace  environment,  adhering  to  the 
constraints  on  aircraft  maneuvering,  and  flight  paths,  and  the  characteristics  of  the 
aerospace  environment.  Few  have  examined  these  influences  in  the 
unconstrained  3D  "electronic  space"  of  the  virtual  world.  Hence,  caution  must  be 
exercised  in  extrapolating  the  conclusions  reached  to  the  issues  of  data 
visualization  in  virtual  environments. 

Nevertheless,  certain  hypotheses  may  be  voiced  regarding  which  features 
will  be  more  effective  for  which  subtasks.  In  order  to  provide  some  concrete 
context  for  interpreting  these  hypotheses,  we  first  describe  briefly  the  paradigm 
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that  we  will  use.  Subjects  will  be  asked  to  navigate  through  a  3D  volume,  filled 
with  data  points,  visiting  each  point  in  turn.  The  desired  target  for  a  given  leg  will 
be  specified  by  flashing,  as  soon  as  the  previous  target  is  encountered.  The  volume 
of  points  are  not  randomly  distributed,  but  are  configured  to  conform  to  a 
particular  3D  volume  or  shape  (e.g.,  surface,  string,  varied  density,  etc.). 
Periodically  subjects  will  be  probed  regarding  the  specific  position  of  certain 
objects  and,  after  each  configuration  of  data  points  has  been  traversed,  subjects 
will  be  asked  about  its  general  form.  Hence,  we  may  characterize  the  task  by  its 
phases  of  search,  travel,  local  judgments  (object  location),  and  global  judgments 
(configuration  description). 

We  anticipate  that  search  will  be  facilitated  by  displays  that  can  present  the 
whole  domain  within  view  (exocentric  and  co-planar),  and,  on  the  basis  of  the 
study  by  Zenyuh  et  al.,  will  be  helped  by  stereo.  The  research  fi-om  the  aviation 
domain  clearly  predicts  that  TRAVEL  will  be  best  supported  by  the  egocentric  3D 
display  (Wickens  and  Prevett,  1995;  Haskell  and  Wickens,  1993),  whereas  strong 
differences  in  favor  of  either  the  coplanar  display,  or  the  exocentric  3D  display  are 
not  predicted,  given  the  approximate  equivalence  of  these  two  formats  in  prior 
research.  Thus,  we  anticipate  that  the  scanning  costs  of  the  co-planar  format  may 
be  traded  off  against  the  ambiguity  costs  of  the  exocentric  3D  format. 

Predictions  regarding  local  judgments  are  not  strong  either,  given  the 
possibility  of  these  tradeoffs.  Since  all  local  judgments  will  be  based  upon  objects 
that  are  visible  in  the  forward  field  of  view  of  the  immersed  display,  and  judgments 
involved  "closest  to,"  rather  than  "left-right,"  we  do  not  have  reason  to  predict  a 
difference  between  the  two  3D  displays.  We  do  hypothesize  that  3D  display 
judgments  will  be  faster  (because  of  reduced  scanning),  but  less  accurate  (because 
of  ambiguity),  in  contrast  to  the  2D  displays. 

Finally  we  predict  a  substantial  superiority  for  the  3D  exocentric  display  in 
terms  of  the  support  for  global  judgments  of  configuration  shape.  This  display 
suffers  neither  the  deficiency  of  keyhole  viewing  (of  the  immersed  display),  nor 
that  of  integration  across  panels  (of  the  co-planar  display).  Wickens,  Merwin,  and 
Lin's  (1994)  finding  of  a  3D  advantage  for  integration  questions  provides  one 
empirical  basis  for  the  prediction  of  this  superiority,  in  a  paradigm  using  materials 
similar  to  that  in  the  present  experiment. 


19 


METHOD 


Subjects 

Thirty  students  at  the  University  of  Illinois  (15  males  and  15  females),  all 
with  normal  or  corrected  to  normal  vision,  served  as  voluntary  participants.  All  of 
the  subjects  were  paid  $5.00  an  hour  for  their  participation. 

Two  groups  for  the  between  subjects  manipulation  of  FOR  were  formed 
by  assigning  eight  males  and  seven  females  to  one  group  and  conversely  seven 
males  and  eight  females  to  the  other  group.  Other  than  the  separation  based  upon 
gender,  the  assignment  was  completely  random. 

Apparatus  and  Displays 

All  subjects  were  required  to  execute  a  navigation  task  rendered  in  3D 
space  on  a  Silicon  Graphics  IRIS  workstation.  There  were  three  displays 
perspectives  used  to  guide  the  subjects  in  the  3D  space;  2D,  immersed  and  non- 
immersed.  Figures  1,  2  and  3  show  prototypical  examples  of  the  2D,  3D 
immersed,  and  3D  non-immersed  perspectives,  respectively. 

Initially  all  subjects  were  to  participate  in  three  types  of  trials,  2D,  3D 
monoscopic  and  3D  stereoscopic  (for  later  subjects,  the  2D  condition  was 
deleted  for  reasons  that  will  be  explained  below).  Stereoscopic  viewing  was 
produced  by  the  use  of  Silicon  Graphics  Stereoview  glasses. 

Procedure 


All  subjects  were  instructed  to  navigate  the  3D  space  as  quickly  and  with 
as  little  joystick  input  as  possible.  The  actual  data  environment  was  represented  as 
a  cube  volume  with  six  different  colored  walls.  The  volume,  with  6  uniquely 
colored  walls,  contained  a  series  of  15  major  objects  (destinations)  along  with  200 
randomly  placed  smaller  objects.  The  15  major  objects  were  defined  within  each 
cube  to  represent  a  particular  3D  pattern.  The  description  of  this  pattern  was  the 
goal  of  the  subjects  in  the  global  judgment  task. 

The  actual  task  required  the  subject  to  navigate  the  target  icon.  This  icon 
was  represented  by  a  small  arrow  head  symbol  in  one  (3D)  or  both  (2D)  panels  of 
the  non-immersed  conditions  and  by  the  display  viewpoint  defining  the  subject’s 
field  of  view  for  the  immersed  condition.  A  joystick  was  used  to  make  pitch  and 
yaw  adjustments.  Constant  deflection  angles  on  either  axis  produced  constant  rate 
of  change  of  yaw  or  pitch.  Forward  motion  was  induced  by  means  of  pressing  the 
joystick  button.  This  provided  the  motion  capability  necessary  to  navigate  through 
the  3D  space  along  a  path  orientated  toward  the  flashing  target  cubes.  A 
particular  cube  within  the  environment  would  flash,  indicating  its  status  as  the 
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present  target  cube.  After  the  subject  successfully  navigated  the  icon  to  intercept 
the  target  cube,  its  flashing  would  cease  and  a  new  target  cube  would  begin  to 
flash.  Once  search  determined  the  location  of  the  next  cube,  the  subject  would 
then  orient  and  navigate  the  icon  into  the  new  target  cube  (recall  that  for  the 
immersive  conditions,  search  and  orientation  were  done  in  parallel).  Steering 
corrections  could  be  made  at  any  time  (with  or  without  forward  movement).  The 
program  would  automatically  terminate  that  particular  trial  after  interception  of  the 
15th  target  cube.  In  this  manner  a  complex  3D  path,  reaching  all  15  targets,  was 
traversed  through  the  volume. 

During  each  navigation  trial  the  program  would  periodically  halt  to  ask  the 
subject  to  make  a  precise  relative  judgment  of  the  location  of  an  object  within  the 
current  field  of  view  (as  defined  for  the  immersive  condition);  identical  questions 
were  asked  for  the  2D  and  3D  non-immersive  displays.  This  would  take  the  form 
of  a  multiple  choice  question  asking  the  subject  which  of  the  two  objects  was 
closer  to  his  or  her  own  position  or  to  a  certain  colored  wall. 

Furthermore,  after  each  navigation  trial  had  been  completed  the  program 
also  asked  a  single  multiple  choice  question  to  assess  the  subjects’  knowledge  of 
the  global  pattern  or  distribution  of  the  particular  data  points  just  encountered. 

This  assessment  began  by  blanking  out  the  entire  screen  and  asking  the  subject  a 
question  regarding  the  general  shape  of  the  data  base  (e.g.,  position  of  target 
cubes,  clumping  of  data  points,  or  shapes  formed  by  the  collection  of  data  points). 

Experimental  Design 

Initially  all  subjects  were  to  complete  a  total  of  15  trials  composed  of  5 
trials  of  each  of  the  three  possible  perspectives:  2D  split  screen  perspective,  a 
monoscopic  perspective  and  a  stereoscopic  perspective.  The  later  two 
perspectives  were  represented  in  either  an  immersed  or  non-immersed  frame  of 
reference,  depending  on  the  experimental  group  to  which  a  subject  was  assigned. 

A  counterbalanced  design  created  6  possible  orderings  of  the  three  possible 
representations  and  was  used  to  reduce  possible  practice  effects  across  trials. 

Variables  Measured 

Data  recorded  for  each  subject  consisted  of  actual  navigation  time  for  each 
trial,  the  accuracy  and  speed  on  both  the  relative  judgment  and  global  judgment 
questions,  the  average  magnitude  of  joystick  inputs  required  to  find  the  next  target, 
and  the  average  magnitude  of  joystick  inputs  required  to  travel  toward  the  next 
target.  These  inputs  were  measured  in  two  ways.  First,  search  inputs  were 
defined  as  the  amount  of  X  and  Y  axis  joystick  movements  recorded  from  the 
moment  the  craft  intercepted  the  present  target  cube  until  the  subject  accelerated 
toward  the  next  target  cube.  Secondly,  travel  inputs  were  defined  as  the  amount 
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of  X  and  Y  axis  joystick  movements  recorded  from  the  moment  the  craft 
accelerated  toward  the  next  target  cube  until  it  penetrated  the  next  target  cube. 
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RESULTS 


The  analysis  for  this  experiment  could  not  proceed  according  to  a  mixed 
ANOVA  model.  This  constraint  is  due  to  the  fact  that  the  design  was  not  fully 
crossed.  The  frame  of  reference  (FOR)  viewing  condition  was  not  defined  for  the 
2D  condition  and  thus  both  FOR  (immersed  and  non-immersed)  subject  groups 
used  the  same  2D  displays  to  completed  the  task.  Therefore,  the  analysis  including 
the  2D  condition  was  conducted  separately  from  the  stereoscopic  vs.  monoscopic 
analysis.  For  clarification,  the  first  analysis  consisted  of  a  one  way  ANOVA  which 
compared  the  three  frames  of  reference  against  one  another  (2D,  immersive  and 
non-immersive),  the  latter  two  only  within  the  monoscopic  viewing  condition.  In 
terms  of  Figure  4,  the  FOR  analysis  for  main  effects  compared  performance  in 
Areas  1,  2,  and  4.  As  a  continuation  of  the  first  (FOR)  analysis,  two  Tukey  post- 
hoc  tests  were  conducted  on  any  significant  main  effects.  The  first  Tukey  test  used 
the  standard  alpha  level  of  .05.  The  second  Tukey  test  was  aimed  at  finding 
marginally  significant  differences  and  used  an  alpha  level  of  .10.  In  the  write  up  of 
the  post-hoc  analysis  all  significant  differences  are  at  an  alpha  level  of  .05  and  all 
non- significant  differences  are  at  an  alpha  level  of .  10. 

The  second  analysis  was  a  2X2  design  which  analyzed  the  effects  of 
stereoscopic  viewing  across  the  two  frames  of  reference  (immersed  vs.  non- 
immersed)  in  which  stereo  was  manipulated.  Due  to  the  fact  that  the  2D  condition 
was  not  included  in  the  stereoscopic  manipulation,  this  2X2  analysis  was  not  used 
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Figure  4.  Representation  of  experimental  design. 


to  analyze  frame  of  reference  effects  (The  FOR  analysis  accomplished  this),  but 
was  used  only  to  analyze  the  effects  of  stereo  across  the  immersed  and  non- 
immersed  frames  of  reference.  In  terms  of  Figure  4,  the  effects  of  stereo  were 
analyzed  by  comparing  Areas  2,  3,  4,  and  5.  Both  ANOVA’ s  were  analyzed  using 
the  Statistical  Analysis  Software  (SAS)  package. 
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Another  modifying  factor  that  affected  the  analysis  of  this  experiment  was 
the  large  number  of  subjects  who  withdrew  during  experimentation.  The  2D 
display  condition  proved  to  be  so  difficult  and  confusing  in  some  phases,  that  eight 
of  the  15  subjects  who  encountered  the  2D  display  withdrew  from  the  experiment. 

Seven  subjects  (5  males  and  2  females)  completed  the  2D  trials  along  with 
the  other  two  conditions.  Preliminary  analysis  of  these  subjects’  results  indicated 
that  their  performance  was  substantMy  worse  in  the  2D  condition,  especially  in 
terms  of  time  needed  to  complete  the  task.  Therefore,  the  remaining  23  subjects 
were  not  assigned  to  perform  the  2D  trials. 

The  following  sections  present  the  results  of  this  experiment  in  terms  of  the 
form  visualization  subtasks:  Search,  Travel,  Local  Judgment  Support,  &  Global 
Judgment  Support. 

Search 


Search  performance  is  represented  in  terms  of  search  time,  X  axis 
deviations,  and  Y  axis  deviations.  These  variables  were  affected  only  by 
dimensionality  and  frame  of  reference.  There  were  no  interaction  between  any  of 
the  variables. 

Search  time.  The  average  time  required  to  find  the  next  target  is  shown  in 
Figure  5  (note  that  there  is  a  single  data  point  for  the  two  graphs  in  the  2D 
condition.  This  reflects  the  fact  that  only  the  mean  value  of  the  two  groups  is 
plotted.  The  first  ANOVA  (designated  for  FOR  effects)  revealed  a  significant 
main  effect  F(2,4863)  =  436,_p  <.0001,  indicating  faster  search  for  the  non- 
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Figure  5,  Average  time  during  search. 
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immersive  group.  The  FOR  Tukey  analysis  indicated  that  the  non-immersed 
condition  had  a  significantly  (200  msec)  faster  search  time  than  the  immersed  and 
2D  conditions.  Furthermore,  the  immersed  condition  was  also  significantly  faster 
than  the  2D  condition.  The  second  ANOVA  (designated  for  stereo  effects  and 
examining  areas  2  through  5)  revealed  that  the  non-stereo  condition  had  a 
significantly  faster  search  than  the  stereo  condition  F(l,  4423)  =  50.48,  p<  .0001 
(see  Figure  5). 

Search  X-axis  deviations.  Average  deviations  in  the  X-axis  are  shown  in 
Figure  6.  The  first  analysis  revealed  a  mam  effect  of  FOR  F(2,4947)  =  12.88, 
P<.0001,  indicating  that  immersed  viewing  conditions  had  the  highest  X-axis 
deviations.  The  post-hoc  Tukey  analysis  revealed  that  the  immersed  conditions  had 
a  significantly  higher  X-axis  deviations  than  the  non-immersed  and  2D 
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Figure  6.  Average  X  input  during  search. 

conditions.  While  the  trend  indicated  that  the  non-immersed  condition  had  higher 
X-axis  deviations  than  the  2D  condition,  the  difference  was  not  significant.  The 
second  analysis  revealed  that  stereo  conditions  had  significantly  larger  X-axis 
deviations  than  did  the  non-stereo  condition  F(l,4498)  =  128.44,  p<.0001  (see 
Figure  6). 

Search  Y-axis  deviations.  The  deviations  in  the  Y-axis  indicate  a  pattern 
similar  to  the  x-axis  deviations  (see  Figure  7).  The  first  analysis  revealed  a  main 
effect  ofFORF(2,4941)  =  59.29,  p<.0001,  indicating  that  immersed  viewing 
conditions  had  the  highest  Y-axis  deviations.  The  FOR  post-hoc  analysis  revealed 
that  the  immersed  conditions  had  a  significantly  higher  Y-axis  deviations  than  the 
non-immersed  and  2D  conditions.  While  a  trend  indicated  that  the  2D  display  had 
higher  Y-axis  deviations  than  the  non-immersed  display,  the  difference  was  found 
to  be  nonsignificant. 
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Figure  7.  Average  Y  input  during  search. 


The  analysis  for  stereoscopic  effects  (see  Figure  7)  revealed  a  marginally 
significant  (F  (1,4495)  =  3.02,  p  =  .08)  increase  in  Y-axis  deviations  for  the  stereo 
over  the  non-stereo  condition. 

Travel 


Performance  in  successfully  traveling  from  one  target  to  another  was 
measured  in  terms  of  travel  time,  X-axis  deviations,  and  Y-axis  deviations.  While 
the  Y-axis  deviation  measure  was  affected  only  by  FOR  and  stereoscopic  viewing, 
the  time  measure  and  X-axis  deviation  measures  were  influenced  by  a  significant 
interaction  between  FOR  and  stereoscopic  viewing. 

Travel  time.  The  average  time  required  to  travel  from  one  target  to  the 
next  is  shown  in  Figure  8.  The  FOR  analysis  showed  a  main  effect  F(2,4860)  = 
544,  p<.0001,  indicating  that  the  immersed  viewpoint  had  the  fastest  travel  times. 
The  FOR  post-hoc  analysis  revealed  that  the  immersed  condition  had  a 
significantly  faster  travel  time  than  the  immersed  and  2D  conditions.  Furthermore, 
the  non-  immersed  condition  was  also  significantly  faster  than  the 
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2D  condition.  The  analysis  for  stereoscopic  effects  failed  to  show  a  main  effect  in 
travel  time  (T  =  1.35,  p  =  .25).  However,  this  analysis  must  be  considered  within 
the  context  of  the  significant  interaction  between  FOR  and  stereoscopic  viewing 
(F(l,4415)  =  59.29,  p<.0001),  which  indicated  that  the  benefits  of  immersion  (to 
travel  time)  were  diminished  in  the  stereoscopic  condition  (see  Figure  8). 

Travel  X-axis  deviations.  Average  deviations  in  the  X-axis  during  travel 
are  shown  in  Figure  9.  The  repeated  measures  ANOVA  of  FOR  showed  a  main 
effect;^(2,4879)  =  520,  p<.0001,  indicating  that  immersed  viewpoint  had  the 
smallest  number  of  X-axis  deviations.  The  FOR  Tukey  analysis  revealed  that  the 
non-immersed  condition  had  significantly  higher  X-axis  deviations  than  the 
immersed  and  2D  conditions.  Furthermore,  the  2D  condition  was  found  to  have 
significantly  higher  X-axis  deviations  than  the  immersed  condition.  The  analysis 
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Figure  9.  Average  X  input  during  travel. 

for  stereoscopic  effects  (see  Figure  9)  indicates  a  main  effect,  F(l,443)  =  126.57, 
P<.0001,  revealing  that  stereo  conditions  had  significantly  larger  X-axis  deviations. 
The  significant  interaction  between  FOR  and  stereoscopic  effects,  F(  1,443 2)  = 
6.71,^<.009,  mediates  the  effects  of  both  variables.  Specifically,  subjects  using 
the  non-immersed  FOR  showed  a  greater  increase  in  X-axis  deviations  when  using 
stereo  than  did  those  using  the  immersed  FOR. 

Travel  Y-axis  deviations.  Average  deviations  in  the  Y-axis  were  recorded 
from  joystick  input  during  travel  and  are  shown  in  Figure  10.  The  FOR  ANOVA 
indicated  a  main  effect  of  FOR  F(2,4902)  =  149.25,  p<.0001,  revealing  that  the 
immersed  viewing  condition  had  the  smallest  average  Y-axis  deviations.  The  post- 
hoc  FOR  analysis  indicated  the  immersed  condition  had  a  significantly  lower 
magnitude  of  Y-axis  deviations  than  the  non-immersed  and  2D  conditions.  There 
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was  no  significant  difference  in  Y-axis  deviations  between  the  2D  displays  and  the 
non-immersed  displays.  The  stereoscopic  analysis  also  revealed  a  main  effect  of 
stereo  F(l,4456)  =  179.18,_p<.0001,  revealing  the  stereo  condition  to  have 
significantly  higher  Y-axis  deviations. 

Local  Judgment  Support 

Local  judgment  accuracy.  The  average  accuracy  of  local  judgment  tasks  is 
shown  in  Figure  1 1.  Since  there  were  two  possibilities  for  answers  in  the  multiple 
choice  questions  used,  chance  accuracy  was  taken  to  equal  50%  correct.  The  FOR 
analysis  showed  a  main  effect  F(2,327)  =  28.90,  p<.0001,  revealing  that  the  2D 
displays  were  the  least  accurate  viewing  condition.  The  FOR  Tukey  analysis 
indicated  that  the  2D  condition  had  significantly  lower  accuracy  than  did  the  non- 
immersed  and  immersed  conditions.  There  was  no  significant  difference  in 
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Figure  1 1 .  Accuracy  on  local  judgments. 

accuracy  between  the  non-immersed  condition  and  the  immersed  conditions.  A 
similar  analysis  of  stereo  effects  revealed  that  performance  in  the  stereo  conditions 
was  significantly  more  accurate  than  in  the  non-stereo  conditions  F(l,296)  =  3.86, 
E<  .05. 
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Local  judgment  response  time.  Average  times  required  to  respond  to  local 
judgment  questions  are  shown  in  Figure  12.  The  FOR  ANOVA  indicated  a 
significant  main  efiect  F(2,324)  =  12.87,  £<.0001,  revealing  that  the  2D  displays 
had  by  far  the  slowest  response  time  to  the  local  judgment  questions.  The  FOR 
post-hoc  analysis  revealed  that  the  2D  condition  was  significantly  slower  than  the 
non-immersed  and  immersed  conditions.  There  was  no  significant  difference  in 


Figure  12.  Response  time  on  local  judgments. 

response  time  between  the  non-immersed  and  immersed  FORs.  The  stereoscopic 
ANOVA  revealed  no  significant  difference  between  the  two  viewing  conditions  (F 
=  .44,  E  =  .50). 


Global  Judgment  Support 

Global  judgment  accuracy.  Accuracy  on  the  global  judgment  tasks  is 
shown  in  Figure  13.  Since  there  were  four  alternatives  for  each  multiple  choice 
question,  chance  accuracy  is  at  25%  correct.  The  FOR  analysis  revealed  a 
significant  main  effect  F(2,327)  =  34.49,  £<.0001,  indicating  that  the  non- 
immersed  displays  had  the  highest  global  judgment  accuracy.  The  FOR  Tukey 
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Figure  13.  Accuracy  on  global  judgments. 
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analysis  indicated  that  the  non-inunersed  condition  had  a  significantly  higher 
accuracy  than  did  the  2D  and  immersed  conditions.  There  was  no  significant 
difference  in  accuracy  between  the  2D  displays  and  the  immersed  displays.  The 
stereoscopic  ANOVA  revealed  no  significant  difference  in  global  judgment 
accuracy  (P  =  -29,  g  =  .59). 

Global  judgment  response  time.  Average  times  required  to  respond  to 
global  judgment  questions  are  shown  in  Figure  14.  The  FOR  ANOVA  revealed  no 
significant  main  effect  (F  =  .97,  p  =  .38).  The  stereoscopic  ANOVA  revealed 


Figure  14.  Response  time  on  global  judgments. 

stereoscopic  viewing  to  promote  significantly  faster  response  times  than  did  non- 
stereoscopic  viewing  F(l,285)  =  5.45,_p<.02. 
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DISCUSSION 


The  present  experiment  was  intended  to  examine  three  “features  of  virtual 
reality”  (Wickens  and  Baker,  1995)  in  an  effort  to  assess  the  extent  to  which  these 
could  contribute  to  effective  visualization  of  a  3D  volume  of  data,  characteristic  of 
that  confronted  by  the  geologist  who  wishes  to  assess  the  nature  and  configuration 
of  toxic  deposits.  The  three  features,  display  dimensionality,  frame  of  reference, 
and  stereo  viewing  have  each  been  examined  in  a  wide  variety  of  different  task 
environments.  Several  studies  in  particular  have  examined  combinations  of  these 
features  in  aviation-relevant  tasks,  and  their  results  have  some  relevance  for 
interpreting  the  present  data.  However,  missing  from  most  empirical  studies  in  the 
visualization  area  are  the  combined  manipulations  of  these  features  in  a  dynamic 
interactive  environment,  while  operators  perform  a  task,  whose  assessment 
characterizes  success  in  understanding  the  structure  of  the  data. 

In  our  discussion,  we  shall  consider  these  three  features  of  virtual  reality  in 
turn,  as  they  are  expected  to  influence  performance  on  four  tasks  characteristics  of 
users  of  virtual  environment  technology:  search  for  important  objects,  travel  to 
those  objects,  rendering  judgments  about  local  features  in  the  environment,  and 
most  important,  rendering  judgments  about  the  global  or  “holistic”  configuration 
of  data  points  within  the  environment  (e.g.,  are  they  arranged  in  a  clustered 
pattern,  are  they  correlated  across  dimensions  of  space,  etc.?). 

Dimensionality.  The  effect  of  dimensionality  was  most  noteworthy  and 
prominent.  Across  all  tasks,  the  best  3D  display  always  supported  better 
performance  than  the  2D  coplanar  display,  and  the  poorest  3D  display  (which  as 
we  saw,  was  task-dependent),  was  never  substantially  worse  than  the  2D  display. 

In  the  several  studies  comparing  2D  and  3D  displays  that  we  have  carried 
out  in  our  lab  involving  aviation,  air  traflBc  control,  and  data  visualization  (e.g., 
Wickens,  1995;  Wickens,  Merwin,  and  Lin,  1994;  Wickens  and  Prevett,  1995; 
Olmos,  Liang,  and  Wickens,  1995;  Wickens,  Liang,  Prevett,  and  Olmos,  1995; 
Boyer  et  al.,  1995;  Wickens,  Campbell,  Liang,  and  Merwin,  1995;  Haskell  and 
Wickens,  1993),  this  is  the  strongest  and  most  consistent  3D  advantage  that  we 
have  ever  found.  The  reason  for  this  advantage  can  be  attributed  to  the  current 
task  which  truly  forced  integration  across  all  three  axes  of  the  data  volume  —  that 
is,  both  the  distribution  of  data  points  (and  therefore  judgments  on  those  data 
points)  as  well  as  the  requirements  for  travel,  involved  an  equal  mixture  of  Ax, 

Ay,  and  Az.  This  circumstance  is  in  contrast  to  many  aspects  of  aviation  and  air 
trafiBc  control,  in  which  the  pilot  and  controller  both  have  reason  to  treat  the  lateral 
aspects  of  flight  differently  and  separately  from  the  vertical  aspects,  both  in  terms 
of  perceptual  judgments  and  control  strategy.  Such  partitioning  has  been  found  to 
lead  to  advantages  for  separated  coplanar  displays  (Haskell  and  Wickens,  1993). 
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In  the  absence,  in  the  current  data,  of  any  explicit  advantage  for  a  coplanar 
display,  the  two  important  costs  of  this  display  configuration  emerged:  the  visual 
scanning  required  between  the  two  display  panels,  which  accounts  for  the 
prominent  2D  slowing  of  many  task  components,  and  the  “mental  gymnastics” 
required  to  integrate  movement  and  position  on  the  two  displays  in  order  to 
discern  3D  patterns  and  trajectories. 

These  results,  in  short,  are  quite  consistent  with  the  proximity  compatibility 
principle  described  by  Wickens  and  Carswell  (in  press),  that  the  task  of  integrating 
across  axes  is  best  served  by  a  display  which  provides  some  degree  of  integration 
in  the  spatial  depiction  of  those  axes  (see  also  Haskell  and  Wickens,  1993; 
Wickens,  Merwin,  and  Lin,  1994  for  corresponding  results  in  aviation  and  data 
visualization,  respectively). 

Frame  of  reference.  The  difference  between  the  immersed,  ego-referenced 
3D  view,  and  the  exocentric  stabilized  field  of  view  provided  evidence  for 
prominent  tradeoffs  between  tasks,  evidence  which  has  been  observed  in  other 
studies  in  aviation  (c.f,  Wickens,  Liang,  et  al.,  in  press;  Wickens  and  Prevett, 
1995).  In  particular,  we  noted  these  tradeoffs  in  three  aspects  of  performance, 
each  of  which  can  be  accounted  for  by  different  information  processing 
mechanisms.  First,  search  was  disrupted  by  the  ego-referenced  frame,  a  disruption 
we  attributed  to  the  “keyhole”  phenomenon  associated  with  the  relatively  narrow 
GFOV  provided  by  this  display.  Since  the  whole  domain  is  not  in  view  at  any 
given  time,  then  looking  for  a  target  not  present  on  the  display,  is  akin  to  swinging 
a  wide  beam  flashlight  around  the  space  —  far  less  efficient  than  a  search  driven  by 
visual  scanning,  characteristic  of  the  non-immersed  display.  The  added  difficulty 
of  the  egocentric  search  caused  by  the  keyhole  is  evidenced  by  the  greater  amount 
of  control  movement  observed  during  the  search  phase  processing.  Of  course  with 
the  egocentric  display,  once  the  target  had  been  found,  the  orientation  phase  was 
already  completed,  whereas  this  still  needed  to  be  accomplished  with  the 
exocentric  display.  But  this  benefit  was  obviously  small,  relative  to  the  “keyhole 
cost,”  leading  to  a  net  loss  of  several  seconds  in  immersion  search  performance. 

In  contrast,  as  a  second  example  of  the  tradeoff,  performance  during  the 
travel  phase  clearly  benefited  fi’om  the  egocentric  FOR,  a  finding  well  replicated  in 
flight  tasks  (Wickens  and  Prevett,  1995;  Barfield  et  al.,  1995).  We  suggest  four 
possible  reasons  for  the  immersion  benefit  to  travel:  (1)  when  the  axis  of  control 
rotation  corresponds  with  the  display  viewpoint  (the  immersed  display),  a  certain 
natural  or  “ecological”  compatibility  is  achieved,  which  readily  supports  ego 
motion  (Warren  and  Wertheim,  1990);  (2)  the  immersed  view,  traveling  through 
the  object-populated  world  provides  the  user  with  a  very  salient  sense  of  ego 
motion  fi’om  the  “flow  field”  of  objects  streaming  passed  the  viewpoint.  This 
optical  flow,  a  cue  of  assistance  for  travel  (Larish  and  Flach,  1990)  is,  of  course, 
missing  fi’om  the  exocentric  view;  (3)  the  exocentric  view  suffers  to  some  extent 
fi’om  ambiguity  along  the  line  of  sight;  that  is,  judgments  of  precise  location  along 
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the  viewing  axis,  relative  to  the  target  cube  are  difficult  when  the  user’s  icon  is 
close  to  the  cube;  and  (4)  a  final  source  of  exocentric  cost  probably  relates  to  the 
fact  that  we  chose  a  static  view  point.  This  meant  that  on  trajectories  moving 
back  toward  the  viewer,  what  movements  that  were  to  the  left  on  the  display 
required  rightward  corrections  on  the  control  and  vice  versa.  That  is,  either  a 
control  reversal  strategy,  or  some  form  of  mental  rotation  was  required  for  control 
in  this  direction,  features  that  we  have  established  to  be  clear  sources  of  difficulty 
with  fixed  map  displays  (Aretz  and  Wickens,  1992;  Wickens,  Liang,  Prevett,  and 
Olmos,  in  press),  and  probable  sources  of  the  added  (less  efficient)  control 
activities  observed  with  the  exocentric  display. 

Turning  to  the  two  judgment  tasks,  we  found  a  modest,  but  reduced  (2 
secs)  immersion  benefit  to  local  judgments  time,  but  no  advantage  for  accuracy  (in 
fact,  there  was  a  nonsignificant  10%  loss).  Such  judgments  were  not  of  course 
hindered  in  the  immersed  view  by  keyhole  effects,  since  the  requested  attributes 
were  always  visible  on  the  display  panel.  In  contrast,  for  the  task  that  may  be 
thought  of  as  most  critical  for  data  visualization  --  the  recall  of  global 
configuration  -  the  exocentric  view  realized  a  substantial  advantage  in  both  the 
time  and  the  accuracy  of  response.  In  this  regard,  our  results  were  quite  consistent 
with  those  of  Wickens  and  Prevett  (1995)  who  also  found  exocentric  advantages 
to  measures  of  spatial  awareness.  However,  we  attribute  the  source  of  this 
advantage  to  a  somewhat  different  mechanism  than  that  inferred  by  Prevett  and 
Wickens.  We  consider  in  the  current  study  that  the  difficulties  with  global 
awareness  resulted  from  problems  with  forming  a  “mental  picture”  of  the 
configuration  from  several  sequential  keyhole  views  (not  unlike  the  2D-3D  effect 
on  global  awareness;  2D  requires  integrating  views  separated  in  space,  not  time). 
This  challenge  was  not  the  cause  of  the  deficiency  observed  by  Wickens  and 
Prevett,  since  the  GFOV  in  their  display  was  always  adjusted  to  be  the  same  for 
immersed  and  nonimmersed  views. 

Stereo  effects.  The  presence  of  stereo  viewing  conditions  has  been  found, 
in  past  research,  to  aid  depth  and  distance  judgments  by  augmenting  the 
impoverished  depth  cues  along  the  LOS  (Wickens,  Todd,  and  Seidler,  1989;  Yeh 
and  Silverstein,  1992;  Sollenberger  and  Milgram,  1993).  Such  evidence  was 
provided  here  when  stereo  was  found  to  assist  conditions  that  suffered  from 
ambiguity  --  in  particular,  the  cost  of  travel  in  the  3D  exocentric  viewing  condition, 
and  the  accuracy  of  local  judgments  for  both  3D  conditions. 

Equally  important  was  the  information  provided  by  the  present  data, 
regarding  where  stereo  was  not  helpful.  First,  it  provided  no  benefit  to  search 
performance.  This  contrast  with  the  results  observed  by  Zenyuh  et  al.  (1988)  can 
be  easily  explained  by  the  fact  that  for  these  investigators,  the  target  attribute  was 
defined,  in  part,  by  the  very  depth  axis  which  stereo  helped  to  resolve.  This  was 
not  the  case  in  the  present  experiment,  since  the  targets  were  defined  by  an 
attribute  --  flashing  -  that  was  unrelated  to  depth..  Second,  stereo  provided  no 
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assistance  to  immersed  travel.  This  finding  is  consistent  with  the  observation  made 
by  Nataupsky  and  Crittenden  (1988)  using  a  flight  control  task,  that  stereo  offered 
no  benefits  to  an  ego-referenced  “tunnel  in  the  sky,”  when  that  display  contained 
other  rich  cues  for  depth,  distance,  and  ego  motion  judgment.  Indeed  these  results 
are  consistent  with  conclusions  drawn  fi’om  their  review  of  the  literature  by 
Wickens,  Todd,  and  Seidler  (1989):  when  ample  depth  cues  are  available  fi'om 
motion  parallax  (as  they  were  here  in  the  ego,  but  not  the  exocentric  displays),  the 
addition  of  stereo  cues  offers  few,  if  any,  benefits. 

Conclusions.  Collectively,  we  may  ask  how  these  results  speak  to  the 
importance  of  the  three  features  of  VR  for  data  visualization.  In  making  such  an 
evaluation,  we  can  consider  two  characteristics,  the  relative  importance  of  each  of 
the  four  subtasks  to  the  full  visualization  objective,  and  the  ease  of  automating 
these  tasks  by  other  means. 

In  this  regard,  we  argue  that  search  and  global  judgments  are  probably 
most  critical.  Furthermore,  neither  of  these  subtasks  can  be  readily  automated 
because  the  scientist  does  not  always  know  what  she  is  looking  for  (search)  till  it 
has  been  found,  and  of  course,  assessment  of  global  configuration  is  often  the  goal 
of  irreplaceable  human  insight. 

On  this  account,  the  exocentric  display,  augmented  where  necessary  by 
stereo,  is  clearly  the  display  of  choice.  The  one  task  in  which  the  egocentric 
immersed  display  excelled  is  one  that  is  not  necessarily  that  important  for 
interactive  data  visualization.  It  may  be  regarded  as  a  means  to  an  end,  rather  than 
an  end  itself.  Furthermore,  travel  is  a  component  that  may  be  automated  with 
relative  ease:  the  user  can  identify  the  desired  destination  of  travel  (e.g.,  by 
positioning  a  cursor  on  the  target),  and  then  allow  the  computer  to  “fly  “  such  a 
trajectory.  Finally,  it  should  be  noted  that  the  exocentric  display  used  in  this  study 
contained  an  element  —  the  fixed  viewpoint  -  which  we  know  to  inhibit 
navigational  travel  (Wickens  et  al.,  in  press).  It  may  well  be  that  if  the  viewpoint  is 
altered  so  that  it  is  “tethered”  behind  the  icon,  still  retaining  the  full  space  in  view, 
then  the  exocentric  deficiencies  of  travel  will  be  eliminated.  This  option  is  the 
subject  of  current  research. 
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CONCLUSIONS 


Summarizing  the  findings  of  this  work,  we  found  that  the  best  performance 
for  search  and  understanding  or  comprehending  3D  data  sets  should  come  fi-om 
3D  displays  using  an  exocentric  fi'ame  of  reference.  Furthermore,  we  found  that 
augmenting  the  3D  exocentric  displays  with  stereo  provides  the  ideal  combination 
of  accuracy  and  speed  for  scientific  visualization  judgment  tasks.  However  as  we 
mentioned  earlier,  scientific  visualization  has  another  component  that  must  be 
addressed:  Database  navigation.  The  exocentric  FOR,  even  with  the  augmentation 
of  stereo,  still  will  require  support  to  overcome  line  of  sight  ambiguities. 

However,  there  are  some  possible  solutions  to  the  navigation  problem  that 
deserve  future  consideration.  The  first  may  be  a  viewing  option  allowing  the  user 
to  alternate  between  egocentric  and  exocentric  fi'ames  of  reference.  This  would 
allow  the  user  to  choose  the  FOR,  either  egocentric  or  exocentric,  best  suited  for 
the  particular  purpose  (navigation  or  supporting  understanding  of  the  environment 
respectively).  Another  option  would  be  to  tether  the  user’s  viewpoint  behind  the 
controlled  icon  (Barfield,  Rosenberg,  &  Furness,  1995;  Wickens  &  Prevett,  1995) 
so  that  the  viewpoint  is  always  egocentricity  oriented.  This  tethering  would 
present  the  user  with  a  more  exocentric  view  of  the  environment,  which  would 
support  better  global  judgments.  However  the  tether  would  align  the  icon  in  a 
fixed  position  behind  the  icon,  thus  allowing  the  user  to  follow  the  icon  and 
removing  much  of  the  3D  ambiguities  of  exocentric  viewing  as  well  as  mental 
rotation  problems.  The  final  option  could  be  to  designate  a  point  destination  (i.e., 
“clicking  the  point”  with  a  mouse  or  other  input  device)  and  allowing  an 
“autopilot”  to  bring  the  icon  to  it.  This  would  remove  the  user  fi'om  the  tracking 
process  altogether,  and  allow  the  user  to  devote  her  or  his  attention  to  the  task  of 
perceiving  and  understanding  the  environment. 

However  in  terms  of  this  research,  due  to  the  great  importance  that  search 
and  global  judgments  have  in  data  visualization,  a  scientist  examining  data  points 
within  a  3D  volume  would  best  be  served  by  an  3D  exocentric  display  that  visually 
presents  the  entire  3D  volume  under  scrutiny.  This  exocentric  display  would  allow 
the  scientist  faster  searches  (locate  and  orient)  for  particular  entities  and  more 
accurate  global  judgments  than  would  be  possible  with  the  2D  displays  and  the 
“key  hole”  view  created  by  the  3D  egocentric  display.  Furthermore,  the  scientist 
should  not  use  stereoscopic  viewing  during  the  search  phases,  due  to  its  slowing 
effects.  However,  the  scientist  would  benefit  from  stereoscopic  viewing  on  global 
judgment  tasks.  Therefore,  a  viewing  option  allowing  the  scientist  to  alternate 
between  stereo  and  monoscopic  viewing  might  be  ideal. 

This  research  supports  the  viewpoint,  suggested  in  the  Introduction,  that 
more  reality  will  not  necessarily  lead  to  better  performance.  The  driving  force 
behind  visualization  is  to  present  data  visually  in  a  way  that  allows  the  scientist  to 
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obtain  insight  regarding  the  data.  While  VR  will  most  definitely  have  an  important 
impact  on  many  fields  of  science,  we  feel  that  a  complete  application  of  all  features 
of  VR  to  the  field  of  scientific  visualization  would  be  an  over  extension.  Proper 
applications  to  scientific  visualization  must  be  guided  by  empirical  studies  wWch 
analyze  the  specific  effects  of  the  individual  components  of  virtual  reality. 
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